Can online scoring really reflects the rankings of products?

Problem description

There are lots of scenarios that we need to do evaluations, for example, online scoring of movies on IMDB or products on Amazon.com. Via the the online scoring, customers can easily judge which items they would like to choose.

However, can this online score really reflect the rankings of quality of homogeneous products?

We find that sometimes our expectations can have influences on our satisfaction of the items. And products with high expectations sometimes are more difficult to generate same level of satisfaction as production with low expectations. (Expectation and Satisfaction are subjective for each individual.)

Example

Take the scenario of E-commerce as an example.

Expectations for buyers can be based on:

• the historical performance of sellers (such as sales);
• the reputation of sellers
• the historical performance and reputation of peers (Same business categories with similar product types);
• the price displayed;

Based on these factors, customers might build their subjective expectations on the qualities of the products.

Thus, the online scoring of the product might be based on the satisfaction which is the result of difference between expectation and true experience.

Notations

There are $n = 2$ sellers who sell the homogeneous products and $m$ buyers. $E_{ij}$ denotes the expectation of buyer $j$ for seller $i$, where $i=\{L,H\}$. We know that $E_{Lj} < E_{Hj}$.

There exist a space of real values of quality for products, $Q$. And the real value of quality for product sold by seller $i$ is $q_{i} \in Q$.We assume $q_{H} < q_{L}$. (Why?)

There are two stages: Pre-purchase and Post-purchase. Buyer $j$ generate $E_{ij}$ for seller $i$ after checking the information about seller $i$ and its product in the pre-purchase stage. Then after true experience of product, buyer $j$ generate the satisfaction $S_{ij}$ and online scoring $R_{ij}$ for seller i in the post-purchase stage.

Satisfaction of buyer $j$ for seller $i$ is $S_{ij}$, which is the result of inconsistence between quality $q_{i}$ and expectation $E_{ij}$. We denoted it by $S_{ij} = f(q_{i} - E_{ij})$.

The online score of buyer $j$ for seller $i$ is $R_{ij}$, which is the result of satisfaction $S_{ij}$ denoted by $R_{ij} = f_{S}(S_{ij})=f_{R}(q_{i} - E_{ij})$.

Assume that $f_{R}$ is a monotonically increasing function.

(Why? There are two assumptions:

• $R_{ij}$ is positive correlated to quality $q_i$;
• $R_{ij}$ is negative correlated to expectation $E_{ij}$;)

However, it seems to be possible that $q_H-E_{Hj} < q_L-E_{Lj}$ because of greater effect of $E_{Hj} > E_{Lj}$, which would result in $R_{Hj} < R_{Lj}$ for buyer $j$.

Explanation

Does this phenomena exist?

In my opinion, this phenomena really exist. Make it more sensible, I use movie as example (Logics is same), before we watch a Chinese low-cost film, we have lower expectation than high-cost Hollywood films. Thus as long as the Chinese low-cost film has a little bit attractive highlights, we would give it a high evaluation, which might be higher than the high-cost Hollywood films.

If so, how to explain it?

My explanation is:

• sometimes our expectation is not only based on quality of items but also the peers’ performance and reputations.
• Even though, buyers would buy product which maximize the expected utility function value. Some uninformative or irrational buyers would still choose products with low reputation and low price. However, online review is short-time reflection on the product and might not present some perspectives of the product, such as durability.

Future

When calculate the integrated score of sellers, we need take into consideration of peers performance and reputation. (Cluster)

The expectation might be measurable via online sentiment analysis.

Hope this is a good research idea.